Boosting Regression via Classiication

Boosting strategies are methods of improving the accuracy of a prediction (a classii-cation rule) by combining many weakerr predictions, each of which is only moderately accurate. In this paper we present a concise analysis of the Freund and Shapire's AdaBoost algorithm FS97] from which we derive a new boosting strategy for the regression case which is an extension of the algorithm discussed in BCP97]. 1 Boosting classiication Classiication refers in general to the problem of predicting a label in a nite set L for each element of a set I of instances according to some relationship between instances and labels that can be thought as an (unknown) deterministic mapping I 7 ! L or as a joint probability distribution over I L; a prediction is then a mapping I 7 ! L whose error is some measure of the discrepancy between predicted and intended label (according the unknown mapping or the joint distribution). When the cardinality of L is 2, the classiication is said to be binary, otherwise is said to be multiclass. The goal of a boosting strategy is to combine a family of simple predictions in order to obtain a nal prediction with smaller error with respect to the simple ones. For computability reasons, the attention is usually restricted to a sample, i.e. a nite number of pairs .i; l/ drawn from I L according some given distribution; furthermore some generalization results based on the Vapnik-Chervonenkis theory VC71] are usually provided (see, for instance FS97]) to relate the error on the sample to the error with respect to the whole set of instances. For this reasons the following formal proofs concern a probability space h; F; Pi and a mapping c : ! f?1; C1g, where is a nite set representing the instance part of the samples and